A memory device such as a cache memory allows a processor to execute instructions faster by storing the most recently used copies of the main memory in the cache lines of the cache memory. The access latency of a program can be reduced if its required instructions or data are stored in the cache lines of the cache memory.
A non-blocking cache memory is used in an out-of-order micro-architecture to avoid stalling of the processor when a cache memory miss event occurs. Dedicated hardware such as fill buffers allows latter load and store requests to be served even if there is a cache memory miss event. The fill buffers may contain a copy of a cache line in any state and are accessed whenever the data cache is accessed during the execution of a load or store request. A memory line of the main memory may be located either in the fill buffers or in the data cache but not simultaneously in both.
The level one cache memory is frequently accessed by the processor and its size and set-associative configuration can affect the performance of the processor. For example, a level one cache memory can have a size of 32 kilobytes, a 8 way set associative configuration and can hold 64 bytes per cache memory line. To access the level one cache memory for a load request, all the 8 ways of a set of the cache memory must be read to determine which way to get the required data.